Method and system for construction project management using photo imaging measurements

ABSTRACT

The present invention is a method and system of small construction project management by way of photo imaging and measurement capture for use by do-it-yourselfers, handymen and small contractors. The method and system operates on mobile computing devices and includes an image recognition system. By performing various imaging based measurements and then processing the resultant data, the method and system produces bills of materials. invoices, and receipts for the necessary tools and materials required by a construction project.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application claims the benefit of co-pending U.S.application Ser. No. 14/625,790 filed on Feb. 19, 2015, which claims thebenefit of U.S. provisional application No. 61/942162, filed on Feb. 20,2014, the entire disclosure of which is incorporated by reference as ifset forth in its entirety herein.

FIELD OF INVENTION

The present invention relates to a method and system for photo imagingand measurement, and more particularly, to a comprehensive smallconstruction project management system.

BACKGROUND

Do-it-yourselfers, handymen and small contractors are frequentlyinvolved with small construction projects that require various raw toolsand materials that need to be obtained for the projects. The purchasingprocess for these raw tools and materials may frequently require variousprecise measurements and calculations to determine the correct bill ofmaterials, which includes the correct list of tools, and a determinationof the required quantities of materials.

There is a need by do-it-yourselfers, handymen and small contractors fora photo imaging and measurement system operating on various computingdevices, both mobile and desktop, that includes an an imagingrecognition system. The photo imaging and measurement system shouldallow users to scan and input various data related to constructionprojects. The system when manipulated would then produce lists ofrequired tools and materials based on the construction project data.

The present invention is a method and system of photo imaging andmeasurement for use by do-it-yourselfers, handymen and smallconstruction contractors. By performing various imaging basedmeasurements and then processing the resultant data, the method andsystem produces lists of tools and materials needed to complete projectbills of materials, invoices, and receipts. The present inventionaccomplishes these objectives.

SUMMARY

A photo imaging and measurement Application (PIM-P Application) operateson a Computing Device. In some embodiments of the invention, theComputing Device may be a Mobile, Desktop, Laptop, or other CPU device.The Computing Device may comprise an iPhone, iPad, Android phone,Blackberry, Personal Computer, etc., but is not limited to theseexclusive examples. In one embodiment of the invention, the PIM-PApplication may be utilized in either offline mode or online mode.

In offline mode, login to and execution of the PIM-P Application occurson the computing device. Once the Application is accessed, the defaultstatus of the system is “offline.” If the choice is made to remainoffline, the Application prompts to perform “limited” system calibrationlocal to the mobile/desktop computing device. In offline mode, images,measurements and data may be collected and stored locally on theComputing Device, but cannot be fully processed until the system statusenters online mode and the various system servers are accessed.

If the choice is made to go “online,” connection to the internet is madethrough a Main Web Server which receives and directs data and processingrequests to the various system servers and relational data bases. ThePIM-P Application is now able to access and integrate with the varioussystem servers and the functional software capabilities, which are theremote Software as a Service (SaaS) platform.

System calibration SaaS processing occurs in the “online” mode, whenrequired data is collected on the Computing Device and relayed to theSystem Calibration Server via the Main Web Server. Data received andprocessed by the System Calibration Server is then stored in the SystemCalibrate RDB according to the calibration process used: either Frame ofReference; Point of Reference; or 3D Scan.

System processes and calculations (SaaS processing) occur in the“online” mode, when required data is collected on the Computing Deviceand relayed to the System Processes Server via the Main Web Server.Project Name, Photo Image, and Dimension measurements are processed bythe System Processes Server, and then stored along with Project GuideURLs and Tools and Materials in the Project Information RDB .

On completing the initial processing of Project Information above, thesystem notifies the PIM-P Application locally on the Computing Device toselect a Project Guide. Selecting a Project Guide link from thedisplayed list, accesses the Vendor eCommerce Site, via the Main WebServer and the Retail Vendor Web Server in order to view the ProjectGuide “How To” video. At this point, the Retail Vendor eCommerce Siteremains visibly open and accessible to the user while other systemprocesses are carried out by the other remote SaaS functions.

The selected Project Guide data and a system prompt to calculate toolsand materials quantities are sent via the Main Web Server to the SystemProcesses Server. Based on the Project Guide video selected, the SystemProcesses Server retrieves the project Dimensions and the recommendedTools and Materials from the Project Information RDB to calculaterequired project Tools and Materials quantities lists. The RequiredTools list prompts the System Process Server to access the PreferredTool Vendor Web Server and Tool Sku RDB via the Main Web Server toretrieve and display in the Preferred Tool Vendor tools matching theRequired Tools list, in the Application on the Computing Device.Required Tools and Materials are now selected for purchase on the RetailVendor eCommerce Site in standard “Shopping Cart” and “Checkout” format.

In one aspect, a method is disclosed that includes: collecting visualdata of a room via a mobile computing device, including calculatingdimensions of at least one aspect of the room based on an interactionwith a touch enabled display of the mobile computing device; displayinga plurality of project resource templates related to improvementprojects; selecting a project resource template in response to a userinput; displaying a plurality of design components based on a selectedproject resource template and calculated dimensions of the at least oneaspect of the room; displaying an augmented reality environment of theroom on the touch-enabled display, the augmented reality environmentconfigured to allow a user to engage with a selected design component tomanipulate a location and orientation of the selected design componentwithin the augmented reality environment; generating a cost estimate anda set of materials to complete a project with the selected designcomponent; and providing an online cart for purchasing the selecteddesign component and set of materials.

A further aspect provides a method comprising collecting visual data ofa room via a mobile computing device, including calculating dimensionsof at least one aspect of the room based on an interaction with a touchenabled display of the mobile computing device; executing a machinelearning algorithm on collected visual data, the machine learningalgorithm configured to select and display a plurality of designcomponents related to a project based on the visual data; and displayingan augmented reality environment of the room on the touch enableddisplay, the augmented reality environment configured to allow a user toengage with a selected design component to manipulate a location andorientation of the selected design component within the augmentedreality environment.

A still further aspect provides a system comprising a memory; and aprocessor coupled to the memory and configured to: collect visual dataof a room via a mobile computing device, including calculatingdimensions of at least one work area of the room based on an interactionwith a touch enabled display of the mobile computing device; display aplurality of project resource templates; receive a selected proj ectresource template in response to a user input; display a plurality ofdesign components based on the selected project resource template andcalculated dimensions of the work area; display an augmented realityenvironment of the room on the touch-enabled display, the augmentedreality environment configured to allow a user to engage with a selecteddesign component to manipulate a location and orientation of theselected design component within the augmented reality environment;generate a cost estimate and a set of materials to complete a projectwith the selected design component; and provide an online cart forpurchasing the selected design component and set of materials.

Other features and advantages of the present invention will becomeapparent from the following more detailed description, taken inconjunction with the accompanying drawings, which illustrate, by way ofexample, the principles of the invention. Both the foregoing generaldescription and the following detailed description are explanatory onlyand are not restrictive of the non-limiting embodiments as claimed.

BRIEF DESCRIPTION OF THE DRAWING(S)

The foregoing Summary as well as the following detailed description willbe readily understood in conjunction with the appended drawings whichillustrate embodiments of the invention. In the drawings:

FIG. 1 is a system diagram depicting the system hardware, software andinterfaces in accordance with an embodiment of the present invention.

FIG. 2A is a flowchart of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 2B is a flowchart of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 2C is a flowchart of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 2D is a flowchart of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 3 is a flowchart of a photo imaging and measurement application inaccordance with an embodiment of the present invention;

FIG. 4 is a flowchart of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 5 is a depiction of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 6 is a flow chart of a machine learning feature in accordance withan embodiment of the present invention.

FIG. 7 is a flow chart of a machine learning feature in accordance withan embodiment of the present invention.

FIG. 8 is a flow chart of a machine learning feature in accordance withan embodiment of the present invention.

FIG. 9A is a depiction of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 9B is a depiction of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 9C is a depiction of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

FIG. 9D is a depiction of a photo imaging and measurement application inaccordance with an embodiment of the present invention.

DETAILED DESCRIPTION

Exemplary embodiments of the present inventions are depicted in thevarious drawing figures. Embodiments may be implemented in manydifferent forms and should not be construed as limited to theembodiments described here. Rather, these embodiments are described sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the embodiments to those skilled in the art.

FIG. 1 is a diagram depicting the system hardware and software inaccordance with an embodiment of the present invention. A photo imagingand measurement Application 105 (PIM-P Application) operates on aComputing Device 100 which may be a Mobile, Desktop, Laptop, or otherCPU device. The Computing Device 100 may comprise an iPhone, iPad,Android phone, Blackberry, Personal Computer, etc., but is not limitedto these exclusive examples. The Computing Device 100 communicates withvarious networked servers via a wireless or hard-wired networkinterface.

The PIM-P Application 105 may be utilized in either offline mode oronline mode. In offline mode, login to and execution of the PIM-PApplication 105 occurs on the Computing Device 100. Once the Application105 is accessed, the default status of the system is “offline.” If thechoice is made to remain offline, the Application 105 prompts to perform“limited” system calibration local to the mobile/desktop computingdevice 100, thus producing Limited Frame of Reference Data 103; LimitedPoint of Reference Data 107; Limited 3D Scan Data 113. In offline mode,images, measurements and data may be collected and stored locally on theComputing Device 100, but cannot be fully processed until the systemstatus enters online mode and the various system servers are accessed.If needed, information stored on the Computing Device 100 may beexported to other computing devices including standard desktop/laptopcomputing devices in standard file formats including but not limited to.jpg, .xls, .doc, .ppt, .csv, .pdf etc., once the system status is“online.”

In online mode login to and execution of the PIM-P Application 105occurs on the Computing Device 100. Once the PIM-P Application 105 isaccessed, the default status of the system is “offline.” If the choiceis made to go “online,” connection to the internet is made through aMain Web Server 110 which receives and directs data and processingrequests to the various system servers (130, 155, 190) and relationaldata bases (135, 160, 193). The PIM-P Application 105, local to theComputing Device 100, is now able to access and integrate with thevarious system servers (130,155, 190) and the functional softwarecapabilities. The Application 105 executes on the mobile device, whilethe remote system servers (130, 155, 190) function as a Software as aService (SaaS) platform. Once the system status is online, anypreviously collected and stored calibration data, images, measurementsetc. local to the Computing Device 100 will be automatically processedby the appropriate server and joined in the corresponding RDB.

System calibration SaaS processing occurs in the “online” mode, whenrequired data is collected on the Computing Device 100 and relayed tothe System Calibration Server 130 via the Main Web Server 110. Datareceived and processed by the System Calibration Server 130 is thenstored in the System Calibrate RDB 135 according to the calibrationprocess used: either Frame of Reference 140; Point of Reference 145; or3D Scan 150.

System processes and calculations (SaaS processing) occur in the“online” mode, when required data is collected on the Computing Device100 and relayed to the System Processes Server 155 via the Main WebServer 110. Project Name 165, Photo Image 170 and Dimension measurements180 are processed by the System Processes Server 155 and then storedalong with Project Guide URLs 185 and Tools and Materials 187 in theProject Information RDB 160.

On completing the initial processing of Project Information above, thesystem notifies the PIM-P Application 105 locally on the ComputingDevice 100 to select a Project Guide. The System Processes Server 155points to the URLs 185 in the Project Information RDB 160 and displaysin the PIM-P Application 105 on the Computing Device 100, the list ofProject Guide “How To” video links that correspond to the Project NameData 165 and Photo Image Data 170. Selecting a Project Guide link fromthe displayed list accesses the Vendor eCommerce Site 120, via the MainWeb Server 110 and the Retail Vendor Web Server 115 in order to view theProject Guide video. At this point, the Retail Vendor eCommerce Site 120remains visibly open and accessible to the user while other systemprocesses are carried out by the other remote SaaS functions.

A Project Guide “How To” video is selected and confirmed in theApplication 105, for processing use. The selection of a Project Guidevideo prompts the system to calculate tools and materials quantities.The prompt is sent via the Main Web Server 110 to the System ProcessesServer 155. Based on the Project Guide video selected, the SystemProcesses Server 155 retrieves the project Dimensions 180 and therecommended Tools and Materials 187 from the Project Information RDB 160to calculate required project Tools and Materials 187 quantity lists.The Required Tools 187 list prompts the System Process Server 160 toaccess the Preferred Tool Vendor Web Server 125 and Tool SKU RDB 128 viathe Main Web Server 110 to retrieve and display in the Preferred ToolVendor products matching the Required Tools 187 list, in the Application105 on the Computing Device 100.

Required Tools and Materials are now selected for purchase on the RetailVendor eCommerce Site 120 in standard “Shopping Cart” and “Checkout”format.

Image Recognition SaaS processing occurs in the “online” mode, whenrequired image and tag data are collected on the Computing Device 100and relayed to the Image Recognition Server 190 via the Main Web Server110. An Object Image 196 and Object Image Tag 199 are processed by theImage Recognition Server 155 and Image Recognition software then joinedin the Image Recognition RDB 193.

FIG. 2A illustrates a flowchart of the photo imaging and measurementapplication. In step 200, the user logs in with user name and passwordto the PIM-P user interface application 105 executing on the ComputingDevice 100. In step 203, after user login, the PIM-P application 105Home screen appears.

In step 205, the Home screen presents the option to Go Online (Yes orNo). Default is off-line. If Online is selected, processing continues atstep 207. If Offline is selected, processing continues at connector A.At step 207, the Main Web Server 110 is accessed. In step 213, The MainWeb Server 110 (FIG. 1), automatically passes control to the SystemCalibration Server 130 (FIG. 1) and System Calibrate RDB 135 (FIG. 1) toperform one or more selected System Calibration procedure(s) (210, 220,225). In some embodiments, the User may choose one System Calibrationprocedure (210, 220, 225). In other embodiments, the User may utilize acombination of System Calibration procedures (210, 220, 225). At step215, the User chooses the system calibration procedure(s) (210, 220,225).

In step 210, the System Calibration Server 130 performs SystemCalibration by Triangulation Frame of Reference 210. In step 220, theSystem Calibration Server 130 performs System Calibration by ReferenceObject Point of Reference 220. In step 225, the System CalibrationServer 130 performs System Calibration by Target Object 3D Scan 225.

In step 227, on completion of System Calibration, the Application 105returns to the PIM-P Application 105 Home screen to input the ProjectName. Saving the Project Name sends information top the System ProcessesServer 155 (FIG. 1) and Project Information RDB 160 (FIG. 1). In step230, on completion of Project Name, a photo image of the project is thenadded by either Capture New Photo Image 240 (FIG. 2B) or else SelectExisting Photo Image 233.

Referring to FIG. 2B, in step 240, a new photo image is captured andsaved, and processing continues at step 243. A photo image may becaptured by various devices including but not limited to a camera deviceintrinsic to the System Mobile Device 100, another mobile device such asa cellular phone or a computing tablet, a camera device independent ofthe disclosed system hardware, or a scanning device which is independentof the disclosed system hardware. The photo image may be transferred tothe system Mobile Computing Device 100 by multiple methods including butnot limited to; standard wireless or wired Internet connectivity,standard device-to-device direct wireless or hardwired connectivity. Thenew photo image is saved to the System Processes Server 155 (FIG. 1) andProject Information RDB 160 (FIG. 1).

In step 233, an existing photo image is selected. If Select ExistingPhoto is chosen, the photo may be selected from a local file on themobile computing device, or selected from the System Processes Server155 (FIG. 1) and Project Information RDB 160 (FIG. 1). Processing nowcontinues at step 243 (FIG. 2B).

In FIG. 2B, (step 243), once a photo image is selected, the Application105 determines if the Photo Image Dimensions are entered. If the PhotoImage Dimensions are entered, then the Application continues at step250. If the Photo Image Dimensions are not entered, the Application 105proceeds to step 247. In step 247, Photo Image Dimensions are inputtedby selecting a dimension to be measured from a User Interface, thentouching a display and “drawing” the length of the dimension beingmeasured. The user may perform multiple touches around the perimeter ofa hole or curved object, and then connect the touch points by drawingbetween them. This will identify to the system that the object beingmeasured is other than a straight line. Completing and saving the imagedimensions sends the information to the System Processes Server 155(FIG. 1) and Project Information RDB 160 (FIG. 1).

In step 250, after completing dimensions capture, from the Home screen,the image will be defined. If Photo Image is defined, the Application105 proceeds to step 257. If Photo Image is not defined, the Applicationproceeds to step 253. In step 253, Define Photo Image means somethinglike “hole” for a hole in a wall or “P-trap” for the plumbing trap undera sink. This Photo Image definition also tags the image for use by theImage Recognition Server 190 (FIG. 1) and Image Recognition RDB 193(FIG. 1). The defined photo is saved as Photo Image Data 170 in theProject Information RDB 160 (FIG. 1). Once the Photo Image has beendefined, the Application 105 proceeds through connector F. Saving theimage definition sends the information to the Image Recognition Server190 (FIG. 1) and the Image Recognition RDB 193 (FIG. 1) via connector Fas well as the System Processes Server 155 (FIG. 1) and ProjectInformation RDB 160 (FIG. 1).

Referring to FIG. 2D, in step 257, if the Photo Image is defined, theApplication 105 prompts the user to enter an Augment Reality (AR) and/orVirtual Reality (VR) Project Design Module (referred to at timesthroughout as, “AR/VR Project Design Module”). In response to enteringthe AR/VR Project Design Module, the Application 105 inputs the PhotoImage and Photo Image Dimensions and retrieves information stored inProject Information RDB 160 (FIG. 1) based on the combination of ProjectName Data 165 and Photo Image Data 170. In some embodiments, the PhotoImage Data is collected within the AR/VR Project Design Module and thephoto image data may comprise a live or recorded video stream displayedon a touch enabled mobile computing device. The Application 105 mayretrieve information that includes, for example, home improvementproject templates, project “How To” guides, and Project Guide URL links185. In response to retrieving information based on Project Name Data165 and Photo Image Data 170, the Application 105 may display theretrieved information on Mobile Computing Device 100. The AR/VR ProjectDesign Module may be configured to enable the user to add one or moredesign components to the Photo Image—such as, e.g., decor, finishing,and furnishings.

Application 105 may implement any known machine learning capability torecognize and identify design components (component of decor, finishing,and furnishings such as, e.g., furniture, shelving, lamps, and lighting,etc.) from Photo Image Data 170. Application 105 may include amachine-learning algorithm configured to utilize a training data setconsisting of text and/or image data associated with a plurality ofdesign components to enable Application 105 to anticipate, guide andfacilitate the design process. Application 105 may include amachine-learning algorithm such as, but not limited to, k-nearestneighbor, naive Bayes classifier, decision tree, linear regression,support vector machines, and neural networks. Application 105 mayinclude a machine-learning algorithm configured to utilize a trainingdata set consisting of a plurality of project categories assigned to aplurality of images of design components. Application 105 may include amachine-learning algorithm configured to analyze one or more geometricfeatures of an image of a design component. For example, the Application105 may employ Point Cloud (PC) data to render and display one or morethree-dimensional (3D) models of design components in Design View ofstep 259. Application 105 may include a natural language processing(NLP) machine-learning algorithm configured to analyze a plurality ofProject Templates to determine commonalities between each ProjectTemplate. Application 105 may include a NLP machine-learning algorithmconfigured to condense a plurality of Project Templates into a generatedProject Template.

In step 258, after entering the AR/VR Project Design Module, theApplication 105 may prompt the user to select a Project Category derivedfrom a plurality of Project Templates. The Project Category may includea broad category of home improvement projects. The Project Category mayinclude a Project Type that is a sub-category of home improvementprojects. In response to selecting a Project Category and a ProjectType, the Application 105 displays one or more available ProjectTemplates on the Mobile Computing Device 100. A Project Template mayindicate various aspects of the selected Project Category and ProjectType necessary to complete the selected project—such as, e.g., methods,tools, materials, time requirements, skills, etc., required to completea home improvement project in a respective Project Template. ProjectTemplates may be accessible via one or more third-party resources—suchas, e.g., Retail Vendor eCommerce Site 120, or a home improvementproject website. For example, the user may select a Project Category of“kitchen” and a Project Type of “cabinets,” to display a plurality ofkitchen cabinet Project Templates. The user may select a kitchen cabinetProject Template that is suitable for the user's needs. If a ProjectTemplate is selected, the Application 105 proceeds to step 259. If aProject Template is not selected, the Application 105 proceeds to step260 to manually calculate a total project estimate.

In step 260, if a Project Template is not selected, the user is promptedto manually calculate a total project estimate. The total projectestimate based, at least in part, on calculations external to the systemthat include anticipated tools and material entered by the user andsaved to the system. Manually entered information saved to the system instep 260 may be sent to the System Process Server 155 (FIG. 1) and theProject Information RDB 160 (FIG. 1). Application 105 subsequentlyproceeds through connector E.

In step 259, after selecting the Project Template, the user is promptedto access a Design View based on the Project Category and ProjectTemplate selected. Design View may include the user interacting with atouch-enabled user interface of Mobile Computing Device 100 byperforming one or more touch events (e.g., click, drag, drop, etc.) onone or more design components (e.g., decor, finishing, furnishings,etc.). Design View may include one or more design components selectedfrom the Project Template, a retail vendor digital catalogues, websites,or other source that includes image data associated with a respectivedesign component. Design View may include displaying one or more designcomponents as a list on Mobile Computing Device 100. Design View mayinclude the user interacting with digital representations of one or moredesign components, and inserting one or more design components into thePhoto Image. Design View may include using a graphical processing unit(GPU) stored at a first location to render digital representations ofone or more design components and stream the image to a Mobile ComputingDevice 100 at a second location different from the first location.Design View may include using ray tracing to render digitalrepresentations of one or more design components. Design View mayinclude generating a list of one or more design components inserted intothe project image. Design View may include using a generated list of oneor more design components to create an ecommerce shopping cartaccessible to the user.

In step 263, after designing the project in Design View, the Application105 prompts the System Processes Server 155 (FIG. 1) to calculate atotal project estimate of the construction project. The calculated totalproject estimate may include the cost of one or more of the following:tools, materials, decor, finishing, furnishings, design components, timerequirements, etc. The calculated total project estimate may include thecost of one or more design components utilized in Design View. The totalproject estimate may include calculating the cost of each designcomponent utilized in Design View. The total project estimate mayinclude calculating the cost of each design component from a pluralityof vendors and comparing the cost of each design component from eachvendor. Application 105 subsequently proceeds through connector H.

Referring to FIG. 2C, the Application enters step 267 from connector I,or else it enters step 270 from connector E. In step 267, theApplication 105 displays system-calculated materials and preferred toolslists. In step 270, using the system-calculated materials and toolsrequirements or the manually entered tools and materials list, theSystem Process Server 155 (FIG. 1) directs the Main Web Server 110(FIG. 1) to the Retail Vendor Web Server 115 (FIG. 1) and Retail VendoreCommerce Site 120 (FIG. 1). Tools and materials are selected and addedto the Shopping Cart 270 and Checked Out 273, on the Retail VendoreCommerce site 120 (FIG. 1). The purchase is completed using anappropriately aligned Vendor Credit account or an independent CreditCard. The project is then saved on the Mobile Computing Device 100, andProject Information Server 160. In step 276, an electronic receipt istransmitted from the Vendor eCommerce site 120. The Application 105 mainsystem process now ends. The Application 105 main system process nowends.

Referring to FIG. 3, the Application 105 flow enters from connector Awhile in Off-line mode. In step 310, while Off-line, the PIM-PApplication 105 provides limited calibration capability, local to theComputing Device 100 (steps 315, 320, 325). At step 310, the Userchooses one or more system calibration procedures (steps 315, 320, 325).In step 315, the Application 105 performs limited System Calibration byTriangulation Frame of Reference 103 (FIG. 1). In step 320, theApplication 105 performs limited System Calibration by Point ofReference object 107 (FIG. 1). In step 325, the Application 105 performslimited System Calibration by Target Object 3D Scan. 113 (FIG. 1). Instep 330, on completion of limited System Calibration, the Application105 returns to the Application 105 home screen (FIG. 1) to input aProject Name.

In step 335, on completion of Project Name input (step 330), a photoimage of the project is then added from a limited source file on theComputing Device 100. The User may select an existing image (step 345)or else capture a new photo image (step 340). In step 340, the Usercaptures a new photo image. A photo image may be captured by, but notlimited to; a camera device which is intrinsic to the System MobileDevice 100 or other mobile device such as a cellular phone or computingtablet; a camera device which is independent of described systemhardware or a scanning device which is independent of described systemhardware. The photo image may be transferred to the system MobileComputing Device 100 by way of but not limited to; standard wireless orhardwire internet connectivity; standard device-to-device directwireless or hardwired connectivity. The Application 105 then proceeds tostep 350. In step 345, the User selects an existing photo image. IfSelect Existing Photo is chosen, the photo may be selected from a localfile on the Computing Device 100 only. The Application 105 proceeds tostep 350. In step 350, the User inputs Photo Image Dimensions byselecting the dimension to be measured from a user interface, thentouching a Computing Device 100 screen and “drawing” the length of thedimension being measured. The User may first touch multiple pointsaround the perimeter of a hole or curved object, then connect the touchpoints by drawing between them. This will identify to the system thatthe object being measured is other than a straight line. Thesedimensions will be saved locally on the Computing Device 100 until theComputing Device 100 is connected online to the System Processes Server155 (FIG. 1) and Project Information Database 160 (FIG. 1).

In step 355, the User is prompted to Go Online. If “no” is answered,then the operation ends, project information is saved locally, and theApplication 105 returns to the Home screen. If “yes” is answered, theApplication proceeds to step 360. In step 360, Define Photo Image is aforced command. This means that no further processing will occur onlineuntil the project image has been defined as outlined previously in step253.

In step 365, once an image is defined Sub-process A (FIG. 3) completes.The Main Web Server is now accessed and enters Sub-process F (FIG. 4).Sub-process F accesses both the Image Recognition Server 190 (FIG. 1)and the Image Recognition RDB 193 (FIG. 1) as well as the SystemProcesses Server 155 (FIG. 1) and Project Information RDB 160 (FIG. 1).

Referring to FIG. 4, step 400 may be entered from either step 253 (FIG.2B) or else from step 365 (FIG. 3, Sub-process A). The defined (tagged)Photo Image is received at the Main Web Server 110 (FIG. 1). The MainWeb Server identifies the tagged Photo Image data (196 , 199), andforwards it to the Image Recognition Server 190 (FIG. 1) and the ImageRecognition RDB 193 (FIG. 1) for processing and storage by ImageRecognition software (steps 405, 410). The Main Web Server also forwardsthe tagged Photo Image to the System Processes Server 155 (FIG. 1) forproject processing (steps 415, 420). Image dimensional calculations arestored with other related project data in the Project Information RDB160 (FIG. 1).

In step 405, the tagged Photo Image is identified and processed by ImageRecognition software. The tagged Photo Image is then stored in the ImageRecognition RDB 193 (step 410). This builds the Image Recognition RDB193 for future improvement and “smart” system functionality via ImageRecognition software capabilities.

In step 415 the tagged Photo Image is sent to the System ProcessesServer 155 (FIG. 1) where name, definition and dimensional details areprocessed and stored for further calculation. In step 420, the taggedPhoto Image is stored and joined with Project details in the ProjectInformation RDB 160, including Name Data 165, Image Data 170, andDimensions Data 180 (FIG. 1). Once the System Processes Server 155 imageprocessing is complete, Sub-process A completes and exits from connectorG and enters the main system flow from connector G at step 257 (FIG.2B).

Referring to FIG. 5, the Application 105 enters from connector H intostep 500. In step 500 the System Processes Server 155 (FIG. 1) retrievesrecommended tools and materials posted in the selected Project Guidevideo. In step 505 the System Processes Server 155 (FIG. 1) retrievesproject Dimensions Data 180 from the Project Information RDB 160(FIG. 1) and calculates anticipated tools and materials quantities basedon selected Project Guide recommendations and actual projectmeasurements. In step 510, based on a recommended tools list, the SystemProcesses Server 155 (FIG. 1) accesses the Preferred Tool Vendor WebServer 125 (FIG. 1) and the Tool SKU RDB 128 (FIG. 1) to retrieve andlist Preferred Tool Vendor products matching recommended items. WhenSub-process H is complete, it re-enters the main system throughconnector I at step 267 (FIG. 2C).

In some embodiments, Application 105 executes a machine learning (ML)algorithm configured to identify home improvement design solutions basedon visual data selected by a user via Mobile Computing Device 100. Theterm “visual data” (also referred to as a “visual data portfolio”) mayinclude an image, plurality of images, video, or plurality of videos,collected via a sensor (e.g., a camera). The ML algorithm may identifyhome improvement design solutions based on one or more attributes ofvisual data selected by the user—such as, e.g., calculated dimensions ofan area to be renovated, identified design components, internet browsermetadata, color schemes, styles, or user feedback. The ML algorithm mayderive from a training data set that includes a plurality of taggedimages stored in Image Recognition RDB 193. Each image of a trainingdata set may include metadata tags assigned by a user, one or moreproperties identified by Image Recognition Software, and/or third-partydata associated with a given image. In response to executing a MLalgorithm on visual data selected by a user, Application 105 may displayone or more design components in an augment reality (AR) or virtualreality (VR) environment via Mobile Computing Device 100—e.g.,Application 105 may display one or more design components in Design Viewof step 259. In response to executing a ML algorithm on visual dataselected by a user, Application 105 may generate a three-dimensional(3D) digital representation of a design component based on one or moreproperties of the design component and a pre-built 3D model scaffold.

In some embodiments, Application 105 generates a panoramic display basedon collected visual data of a given area. The panoramic display mayinclude merging two or more images to yield a single image that depictsaspects of a given area captured by the two or more images. Thepanoramic display may include measurements of one or more aspects of agiven location captured via a camera of a mobile computing device inresponse to a user interaction with a touch enabled display of themobile computing device.

In some embodiments, Application 105 executes a machine learning (ML)algorithm configured to identify home improvement design solutions basedon a projected increase in value yielded by a home improvement project.The ML algorithm may derive from a training data set that includes realproperty market data—such as, e.g., historical property transactions,historical features and/or design components, etc. The ML algorithm mayderive from a training data set that includes text and/or visual data ofone or more online real property listings. The ML algorithm may identifythe addition or removal of one or more design components from a givenproperty over time based on design components identified by imagerecognition software. The ML algorithm may identify a correlationbetween one or more design components and change in property value viahidden layers of a neural network.

In some embodiments, Application 105 executes a machine learning (ML)algorithm configured to identify home improvement design solutions for afirst location of a home based, at least in part, on visual datacollected from a second location of the home. For instance, the MLalgorithm may identify design components, decor, etc., (i.e., themes) ofa first location not under renovation to determine one or morecomplementary design components for a home improvement project of asecond location. Executing the ML algorithm may include, for example, auser collecting visual data of a first room and visual data of a secondroom via a camera. For example, a user captures images of a first room(Room A) and a second room (Room B) of a residential home using a mobilecomputing device. The user identifies Room A as a kitchen underrenovation, and identifies Room B as a living room that is not underrenovation. Application 105 executes a ML algorithm on the capturedimages of Room A and Room B. The ML algorithm identifies two designcomponents of Room B: a leather couch and a mahogany coffee table. TheML algorithm utilizes a training data set to determine complementarycomponents for Room A based on the identified design components of RoomB. Application 105 displays one or more complementary design componentssuggested to include in Room A for the user to engage in an ARenvironment via the mobile computing device.

In some embodiments, Application 105 executes a machine learning (ML)algorithm configured to identify progression of a home improvementproject over time. The ML algorithm may analyze progression of a homeimprovement project to provide feedback and/or suggestions to a userbased on discrepancies between a first image captured at a first pointin time, a second image captured at a second point in time, and/or aprojected image based on an augmented reality (AR) design created by theuser in Design View. In response to identifying a discrepancy,Application 105 may notify a user of the discrepancy and may providesuggestions to remedy the discrepancy. For example, a user engages in ahome improvement project that includes installing kitchen cabinets. Theuser captures a first image of the kitchen at a first point in time(e.g., start of project), a second image of the kitchen at a secondpoint in time (e.g., middle of project), and a third image of thekitchen at a third point in time (e.g., end of project). In the firstimage at the first point in time, the user has not modified the kitchenand enters Design View displayed on a Mobile Computing Device to engagedigital representations of kitchen cabinets to design the kitchen in anaugmented reality environment. While in Design View, the user selectskitchen cabinets that include a first cabinet and a second cabinetrequiring separate installation. Between the first and second point intime, the user installs the first cabinet and proceeds to capture thesecond image of the kitchen. Application 105 executes a ML algorithm onthe second image to identify discrepancies based on the first image,second image, and a projected image created in Design View. In responseto identifying a discrepancy, such as the first cabinet being misalignedwith the position selected in Design View, the Application notifies theuser of the discrepancy and provides suggested solutions to remedy thediscrepancy.

In some embodiments, Application 105 executes a natural languageprocessing (NLP) algorithm configured to process a plurality of projectguides in which each project guide is associated with a project categoryof a plurality of project categories. The NLP algorithm may implementpart-of-speech tagging (POS tagging) to associate one or more discreetterms of each project guide with one or more descriptive tags.Descriptive tags may include one or more aspects associated with a homeimprovement project—such as, e.g., materials, tools, online vendors,cost estimates, labor estimates, etc. The NLP algorithm maycross-reference two or more project guides associated with a respectiveproject category to identify commonalities and/or differences betweenthe two or more project guides of the respective project category. TheNLP algorithm may generate a project template based on identifiedcommonalities and/or differences of two or more project guidesassociated with a respective project category. The NLP algorithm maygenerate a project template that includes two or more alternatives to anaspect of the home improvement project based, at least in part, onidentified differences. The NLP algorithm may calculate a confidencescore associated with the accuracy of one or more descriptive tagsassigned to one or more discreet terms within a project guide.

Referring to FIG. 6 depicting a method 600 of training a machinelearning algorithm based on real property data 601. The method 600starting with collect property data in step 601. Collecting propertydata in step 601 may include obtaining historical data sets associatedwith real estate listings and attributes of each listing such as, forexample, region demographic 601A, consumer profile 601B, transaction601C, and property image 601D. Image recognition in step 603 includesexecuting a function on property image 601D of each item in collectedproperty data 601 to identify components 605. Subsequently, compiletraining data 607 includes assembling the identified components 605,region demographic 601A, consumer profile 601B, transaction 601C andproperty image 601D into a training data set 607 to train a machinelearning algorithm. The compiled training data 607 trains machinelearning (ML) algorithm 609 to generate a ML algorithm capable ofderiving properties and suggestions for future home improvementprojects. Finally, deploy ML algorithm 611 includes enabling theApplication 105 to programmatically access the ML algorithm to executefunctions on a user's visual data.

Referring to FIG. 7, depicting a method 700 in accordance with anembodiment of the present invention. Collect component data in step 701includes compiling home improvement design components from one or morevendors. Generate component profile in step 703 includes identifyingattributes of the design component and compiling the attributes into amachine-readable format (e.g., JSON, XML, etc.). Match existingcomponent profile model in step 705 compares attributes of the componentprofile yielded in step 703 to determine if a three-dimensional (3D)computer-animated design (CAD) model of a different design componentexists in a database. If a matching 3D CAD model does exist, the method700 proceeds to map profile to model match in step 707 that applies anattribute mesh (e.g., color of the component) to a pre-existing 3D CADmodel. If a matching 3D CAD model does not exist, the method 700 proceedto generate a 3D CAD model based on the generated component profiledyielded in step 703.

Referring to FIG. 8, depicting a method 800 in accordance with anembodiment of the present invention. Capture image of finished room instep 801 includes using a camera on a mobile computing device to collecta first visual portfolio. Calculate theme of finished room in step 803includes executing a ML algorithm on the first visual portfolio yieldedin step 801 to calculate a theme based on one or more design componentsinstalled in the finished room. Capture image of project room in step805 includes using a camera on a mobile computing device to collect asecond visual portfolio of a room that is the subject of a homeimprovement project. Identify home design solutions in step 807 mayinclude executing a ML algorithm on the second visual portfolio toidentify one or more design components that match the calculated themeyielded in step 803. Render model in digital environment of step 809includes displaying the one or more design components yielded in step807 on a mobile computing device in an augmented reality environment.

Referring to FIG. 9A which includes a depiction of a photo imaging andmeasurement application in accordance with an embodiment of the presentinvention. FIG. 9A depicts an embodiment of the present invention thatis collecting images of a room in an AR environment 902 in which a touchenabled mobile computing device 900 collects visual data by scanning theroom. As the user scans the room, the user is able calculate dimensionsof various aspects (i.e., work areas 904 ,906) of the room, e.g., thedimensions on the wall where cabinets are to be installed. Dimensionsmay be calculated by pointing the camera at a location and insertingshapes (e.g., a rectangle) onto a surface or selecting points in the ARenvironment using the touch enabled interface. Once the dimensions ofthe desired work areas are established, the user can scan around theroom via the mobile device see the measured work areas where, e.g., newcabinets are to be installed. (Note that the work areas/dimensions shownoutside of AR viewer are for illustrative purposes only—i.e., they wouldonly be seen within the AR environment 902 shown on the device screen.Accordingly, the AR environment is configured to measure and render aset of work areas within a room, which may be captured and stored usingpanoramic imagery.

Referring to FIG. 9B which includes a depiction of a photo imaging andmeasurement application in accordance with an embodiment of the presentinvention. FIG. 9B depicts an embodiment that is in an augmented realityDesign View mode. No design components, or other computer renderedcomponents, have been included yet in the AR environment 902 shown onthe device screen. Shown along the bottom bar of the application are aset of design component options 908 that the user can drag and drop. Thedesign components 908 may for example be selected, ranked and displayedusing a ML or other algorithm. For example, the options 908 may beselected based on the theme of the room or other rooms in the house, thelikely return on investment, user inputs, available stock, dimensions,etc.

FIG. 9C shows a depiction of a photo imaging and measurement applicationin accordance with an embodiment of the present invention. FIG. 9Cdepicts an embodiment that is in an augmented reality Design View mode.A kitchen cabinet design component 910 is selected (i.e., dragged anddropped) by the user and rendered on the wall that was previously emptyas shown in FIG. 9B. During the drag and drop process, the component 910may be initially displayed and manipulated by the user (location andorientation) to be established proximate the workspace on the wall andthen automatically scaled and rendered in an updated 3D rendering asshown. manipulate a of the selected design component within theaugmented reality environment;

Referring to FIG. 9D which includes a depiction of a photo imaging andmeasurement application in accordance with an embodiment of the presentinvention. FIG. 9D depicts an embodiment that is in an augmented realityDesign View mode and has a kitchen cabinet rendered on a mobile devicesimilar to the embodiment of FIG. 9C. In this case, the user has zoomedinto a particular area of the design component 910 within the ARenvironment.

Embodiments of the invention may be practiced as methods, systems ordevices. Accordingly, embodiments may assume the form of a hardwareimplementation, a firmware implementation, an entirely softwareimplementation or an implementation combining software, firmware andhardware aspects. The detailed description here is, therefore, not to betaken in a limiting sense.

Unless specifically stated otherwise here, it is intended thatthroughout the description, terms such as “processing” or “computing” or“calculating” or “determining” or “displaying” or the like, refer to theaction and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (electronic) quantities within the computer system memories orregisters or other such information storage, transmission or displaydevices.

The present invention also relates to an apparatus for performing theoperations herein. This apparatus may be specially constructed for therequired purposes, or it may comprise a general-purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but is not limited to, any type ofdisk including floppy disks, optical disks, CD-ROMs, magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical cards, application specific integratedcircuits (ASICs), or any type of media suitable for storing electronicinstructions, and each coupled to a computer system bus. Furthermore,the computers referred to in the specification may include a singleprocessor or may be architectures employing multiple processor designsfor increased computing capability.

The processes and displays presented herein are not inherently relatedto any particular computer or other apparatus. General-purpose systemsmay be used with programs in accordance with the disclosure here, ormore specialized apparatus may be utilized to perform the requiredmethod steps.

In addition, the present invention is not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the present invention as described herein, and any references tospecific languages are provided for disclosure of enablement and bestmode of the present invention.

Embodiments of the present disclosure are described here with referenceto block diagrams and/or operational illustrations of methods, systems,and computer program products according to embodiments of the presentdisclosure. The functions/acts noted in the blocks may occur out of theorder as shown in any flowchart. For example, two blocks shown insuccession may in fact be executed substantially concurrent or theblocks may sometimes be executed in the reverse order, depending uponthe functionality or procedures involved. Additionally, not all of theblocks shown in any flowchart need to be performed and/or executed. Forexample, if a given flowchart has five blocks containing functions orprocedures, it may be the case that only three of the five blocks areperformed and/or executed. In this example, any of the three of the fiveblocks may be performed and/or executed.

The mobile or desktop computing device 100 may include a WIFI or wirednetwork interface. The Computing Device 100 may consist of any of avariety of electronic devices including but not limited to mobiletelephones, cellular telephones; PDA's equipped with communicationcapabilities, and mobile computers or palm computers and desktoppersonal computers with various wireless or wired communicationcapabilities. The desktop Computing Device 100 may be comprised of anyof the standard devices available including but not limited to deviceswhich support the Apple, Microsoft, or Android operating systems withinterfaces to the Internet. In addition to supporting the functionalityof the present invention, the Computing Device 100 may also providecommon mobile communication functions such as placing telephone calls,email and texting.

It is understood that aspects of the present disclosure may beimplemented in any manner, e.g., as a software program, or an integratedcircuit board or a controller card that includes a processing core, I/Oand processing logic. Aspects may be implemented in hardware orsoftware, or a combination thereof. For example, aspects of theprocessing logic may be implemented using field programmable gate arrays(FPGAs), ASIC devices, or other hardware-oriented systems.

Computer readable program instructions for carrying out operations ofthe present disclosure may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Python, Smalltalk, C++ orthe like, and conventional procedural programming languages, such as the“C” programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present disclosure.

While embodiments of the invention have been described in detail above,the invention is not limited to those specific variations. The ascribedinvention descriptions should be considered as merely exemplaryillustrations set forth for a clear understanding of the principles ofthe invention. Further variations, modifications, extensions, orequivalents of the invention may be developed without departing from thescope of the invention. It is therefore intended that the invention notbe limited to the particular embodiments disclosed as the best modecontemplated for carrying out this invention, but that the inventionwill include all the embodiments falling within the scope of theappended claims.

What is claimed is:
 1. A method comprising: collecting visual data of aroom via a mobile computing device, including calculating dimensions ofat least one aspect of the room based on an interaction with a touchenabled display of the mobile computing device; displaying a pluralityof project resource templates; selecting a project resource template inresponse to a user input; displaying a plurality of design componentsbased on a selected project resource template and calculated dimensionsof the at least one aspect of the room; displaying an augmented realityenvironment of the room on the touch-enabled display, the augmentedreality environment configured to allow a user to engage with a selecteddesign component to manipulate a location and orientation of theselected design component within the augmented reality environment;generating a cost estimate and a set of materials to complete a projectwith the selected design component; and providing an online cart forpurchasing the selected design component and set of materials.
 2. Themethod of claim 1, wherein displaying the augmented reality environmentfurther comprising: displaying the plurality of design components as aselectable set of items; dragging a first digital representation of theselected design component in response to a user input from the set ofselectable items via the touch-enabled display; establishing the firstdigital representation at a first position within the augmented realityenvironment; and rendering an updated digital representation of theselected design component at the first position.
 3. The method of claim2, wherein the updated digital representation renders the selecteddesign component to scale relative to the room within the augmentedreality environment.
 4. The method of claim 1, wherein collecting visualdata of the room includes generating a panoramic image of the room. 5.The method of claim 1, wherein displaying the plurality of designcomponents includes executing a machine learning algorithm that ranksthe plurality of design components.
 6. The method of claim 1, whereincollecting visual data of the room includes capturing an image of theroom via a camera of the mobile computing device.
 7. A methodcomprising: collecting visual data of a room via a mobile computingdevice, including calculating dimensions of at least one aspect of theroom based on an interaction with a touch enabled display of the mobilecomputing device; executing a machine learning algorithm on collectedvisual data, the machine learning algorithm configured to select anddisplay a plurality of design components related to a project based onthe visual data; and displaying an augmented reality environment of theroom on the touch enabled display, the augmented reality environmentconfigured to allow a user to engage with a selected design component tomanipulate a location and orientation of the selected design componentwithin the augmented reality environment.
 8. The method of claim 7,executing a machine learning algorithm further comprises: training themachine learning algorithm, including: collecting a real propertylisting data set from a plurality of real estate listings that includesimages of each listing; processing images of each listing to identifydesign components; assigning a tag for design components identified ineach listing; compiling a real property training data set that includesassigned tags for each identified design component; and running anunsupervised learning module based on the real property training dataset.
 9. The method of claim 7, wherein collecting visual data includescapturing an image of the room via a camera of the mobile computingdevice.
 10. The method of claim 7, wherein collecting visual dataincludes generating a panoramic image of the room.
 11. The method ofclaim 7, wherein displaying the augmented reality environment furthercomprises: rendering a list of the plurality of design components;dragging a first digital representation of the selected design componentin response to a user input from the list of the plurality of designcomponents via the touch-enabled display; establishing the first digitalrepresentation at a first position within the augmented realityenvironment; and rendering an updated digital representation of theselected design component at the first position.
 12. The method of claim11, wherein the updated digital representation renders to scale relativeto the room within the augmented reality environment.
 13. The method ofclaim 7, further comprising: displaying a plurality of project resourcetemplates related to home improvement projects; and selecting a projectresource template in response to a user input.
 14. A system comprising:a memory; a processor coupled to the memory and configured to: collectvisual data of a room via a mobile computing device, includingcalculating dimensions of at least one work area of the room based on aninteraction with a touch enabled display of the mobile computing device;display a plurality of proj ect resource templates; receive a selectedproj ect resource template in response to a user input; display aplurality of design components based on the selected project resourcetemplate and calculated dimensions of the work area; display anaugmented reality environment of the room on the touch-enabled display,the augmented reality environment configured to allow a user to engagewith a selected design component to manipulate a location andorientation of the selected design component within the augmentedreality environment; generate a cost estimate and a set of materials tocomplete a project with the selected design component; and provide anonline cart for purchasing the selected design component and set ofmaterials.
 15. The system of claim 14, wherein displaying the augmentedreality environment further comprises: displaying the plurality ofdesign components as a selectable set of items; moving a first digitalrepresentation of the selected design component in response to a userinput from the set of selectable items via the touch-enabled display;establishing the first digital representation at a first position withinthe augmented reality environment; and rendering an updated digitalrepresentation of the selected design component at the first position,wherein the updated digital representation renders the selected designcomponent to scale relative to the room within the augmented realityenvironment.
 16. The system of claim 14, wherein collecting visual dataof the room includes generating a panoramic image of the room thatincludes a plurality of work areas.
 17. The system of claim 14, whereindisplaying the plurality of design components includes executing amachine learning algorithm that ranks the plurality of designcomponents.
 18. The system of claim 17, further comprising: collectingvisual data via a mobile computing device, including visual data of afirst location and visual data of a second location, and calculatingdimensions of at least one aspect of the first location and one aspectof the second location based on an interaction with a touch enableddisplay of the mobile computing device; executing a machine learningalgorithm on visual data of the first location, the machine learningalgorithm configured to identify a first theme of the first location,and recommend a design component to install in the second location basedon the first theme; and displaying a digital representation of thedesign component in a computer rendered environment based on visual dataof the second location.
 19. The system of claim 18, wherein identifyingat least the first installed design component includes executing animage recognition function derived from a training data set comprising aplurality of design component images.